fix(security): authenticate extension services - #2638

Summary

Authenticate remote supervisor middleware and gateway-interceptor RPCs with short-lived, exact-audience gateway-signed JWTs. Add a shared extension client foundation so both mechanisms use the same bearer rotation and TLS/custom-CA implementation.

Related Issue

Closes #2623

Security Fix

Remote extension services previously could not cryptographically distinguish OpenShell gateway or sandbox-supervisor calls from direct network callers. This change adds authenticated caller identity and policy-constrained token distribution while preserving the existing Ed25519 signing authority and sandbox refresh API.

Severity Assessment

• Impact is high for network-reachable extensions that perform privileged transformations or actions, and medium when trusted network isolation is enforced.
• Exploitability is moderate because exploitation requires extension endpoint reachability but not compromise of gateway signing material.
• Alpha residual risks remain: bearer replay until short expiry and single-key JWKS rotation. Multi-key overlap, mTLS, and stronger replay resistance remain deferred hardening.

Changes

• Add openshell-extension-core, a shared foundation for extension identity, claims, bearer rotation, and TLS/custom-CA transport used by both supervisor middleware and gateway interceptors.
• Mint short-lived, exact-audience JWTs for gateway and sandbox-supervisor callers from the gateway's existing Ed25519 signing authority.
• Distribute policy-authorized credentials to supervisors through RefreshSandboxToken, held in a supervisor-owned store that rotates slots in place.
• Publish verification material at /.well-known/jwks.json and /.well-known/openid-configuration.
• Agree the expected audience through the Describe manifest at startup rather than relying on configuration matching on both sides of the boundary.
• Add allow_insecure_transport per registration for deployments that keep a plaintext endpoint.
• Update architecture, RFC appendices, user documentation, and generated Go bindings.

Review notes

Compatibility. With gateway_jwt configured, extension endpoints use https://; interceptors may also use unix://, middleware may not, because a middleware endpoint has to be reachable from every sandbox supervisor. Deployments that keep a plaintext endpoint set allow_insecure_transport = true on that registration. OpenShell then attaches no credential, refuses to mint one if a supervisor asks, and names the registration in a warning at every gateway startup.

Rotation cadence. Configuration polling runs every 10s while credentials last 15 minutes. Supervisors reuse their installed slots and rotate only when one is missing or has passed four fifths of its lifetime, so a routine poll does not re-run gateway effective-policy resolution or re-mint the gateway token.

Authorization resolution. handle_get_sandbox_config is the single resolver deciding which registrations a sandbox may hold credentials for. A cheaper parallel resolver would duplicate ~120 lines of policy composition and backfill side effects on a security-critical path, where drift fails open.

RFC layout. Extension-authentication detail lives in appendices/extension-authentication.md under RFC 0009 (the shared contract) and RFC 0010 (interceptor differences), leaving both accepted RFC bodies intact. RFC 0010's appendix is currently reachable only by path, since its body has no appendix reference to link from.

Out of scope: the service-side verification SDK (tracked separately; Rust and Python first), a separate signing key per trust domain, and mTLS. Residual risks are recorded in the RFC 0009 appendix.

Testing

• mise run pre-commit passes
• Affected Rust unit and integration suites pass, including gateway, sandbox, shared core, middleware, and interceptor crates
• Workspace and E2E Rust targets pass strict Clippy checks
• Buf lint and generated Go binding checks pass
• Private-CA success, wrong CA, and hostname-mismatch tests added
• Authorization, exact-audience, expiry, rotation, detachment, secret-descriptor, JWKS, and discovery-document tests added or updated
• Explicit-typing, audience-handshake, insecure-opt-out, credential-store, and mint-bound tests added
• Full Docker-backed E2E suite not run; no external extension fixture currently exists in that lane

Checklist

• Follows Conventional Commits
• Commit is signed off (DCO)
• User-facing and architecture documentation updated
• Agent infrastructure inventory synchronized for the new crate

Example branch

The runnable authenticated middleware example is intentionally kept on the separate 2623-authenticated-middleware-example/pimlock branch. Its service-side JWT verifier is a dedicated example module marked for future extraction into an extension-building SDK, keeping inbound verification concerns out of openshell-extension-core. After this PR lands, the branch can be rebased and merged separately or folded into later work. Its verifier must require typ: openshell-ext+jwt and return expected_audience from Describe.

Follow-up: JWKS bootstrap

Verification keys are served once the gateway is up, so a service's first contact still depends on out-of-band provisioning. /.well-known/openid-configuration covers discovery for services that can reach the gateway. The open question is whether to bind the primary listener early and serve only the well-known routes until ready, or to treat operator provisioning of the public key that certgen already writes to disk as the supported bootstrap and leave both endpoints as convenience. The latter matches how Kubernetes handles the same problem and avoids the listener-swap machinery.